Orbital ordering and frustration of p-band Mott-insulators 
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We investigate the general structure of orbital exchange physics in Mott-insulating states of p- 
orbital systems in optical lattices. Orbital orders occur in both the triangular and Kagome lattices. 
In contrast, orbital exchange in the honeycomb lattice is frustrated as described by a novel quantum 
120°-model. Its classical ground states are mapped into configurations of the fully-packed loop 
model with an extra U(l) rotation degree of freedom. Quantum orbital fluctuations select a six-site 
plaquette ground state ordering pattern in the semiclassical limit from the "order from disorder" 
mechanism. This effect arises from the appearance of a zero energy flat-band of orbital excitations. 

PACS numbers: 03.75.Ss,03.75.Nt, 05.50.+q, 73.43.Nq 



Orbital is a degree of freedom characterized by or- 
bital degeneracy and orientational anisotropy. The inter- 
play among orbital, spin and charge degrees of freedom 
gives rise to important effects on metal-insulator transi- 
tions, superconductivity, and colossal magneto-resistance 
in transition metal oxides P, 0, 0] . The progress of cold 
atom physics in optical lattices has provide a new oppor- 
tunity to investigate orbital physics. A major advantage 
of optical lattices is the absence of the Jahn- Teller lattice 
distortion which lifts orbital degeneracy and quenches the 
orbital degree of freedom in solid state systems. 

Orbital physics in optical lattices exhibits different 
features from those in solid state systems 0, H, @, 0, 



lfl 11, UJ, 13, 14, 15, 161. The Hubbard inter- 



action of p-orbital bosons has the ferro-orbital nature 
leading to an "orbital Hund's rule" . This generates a 
class of orbital superfluid states with complex-valued 
wavefunctions breaking time reversal symmetry beyond 
Feynman's celebrated argument of the positive-definitive 
ground state wavefunctions [1, 0, 0] ■ The p-orbital hon- 
eycomb lattice filled with fcrmions provides ap^-orbital 
counterpart of graphene, whose flat band structure dra- 
matically enhances interaction effects and gives rise to 
various charge and bond crystalline orders [llT Il7ll. The 
experiment progress is truly exciting [l3L Il5l. flij |. 
In particular, the meta-stable p-orbital bosonic systems 
have been realized by using the stimulated Raman tran- 
sition to pump bosons into high orbital bands (lrjj . 

In Mott-insulators, the orbital degree of freedom also 
enables super-exchange interaction just as spin does. A 
marked difference between orbital and spin exchanges is 
that the former depends on bond orientation. Orbital ex- 
change physics has been extensi vely inv estig ated in the 



d-orbital t 2g and e g systems 0, [jji [2(| 21, 22 1. How- 



ever, correlation effects in the p-orbital bands in solid 
state systems are typically weak. To our knowledge, the 
p-orbital exchange physics has not been investigated in 
solid state systems. In contrast, the p-orbital systems in 
optical lattices can be easily tuned to the strong correla- 
tion regime, providing an opportunity to investigate new 
orbital physics. A discussion of the p-orbital exchange in 
the honeycomb lattice and the consequential 120° degree 



model was presented by the author in Ref. [23j. 

In this article, we construct the general structure of 
the p-orbital exchange models in optical lattices. Orbital 
orders are found in the square lattice, and also in the tri- 
angular and Kagome lattices which are typical frustrated 
lattices for spin systems. In contrast, strong orbital frus- 
tration occurs in the honeycomb lattice as described by a 
novel 120°-orbital exchange model. The classical ground 
states are closely related to the fully-packed loop repre- 
sentation of the three-coloring model. The "order from 
disorder" mechanism generates a plaquette orbital order- 
ing pattern form quantum orbital fluctuations. 

We begin with the two dimensional ^-orbital Mott- 
insulators with spinless fermions by loading a single com- 
ponent of fermion atoms. Each optical site is approx- 
imated by an anisotropic harmonic potential well with 
frequencies lu z ^> lj x = u> y . Suppose that the filling is 
two fermions per site: one is in the inert s-orbital and 
the rest fills the p X j,-orbitals. The hopping terms in 
the p-bands can be classified as the er-bonding tu and 
7r-bonding t± (typically t\\/t±_ ~> 1)- Due to the orbital 
degeneracy, the on-site interaction for spinless fcrmions 
is still the Hubbard-like as Hi nt — U^2~np x np y . For 
spinless fermions, the leading contribution to U is from 
the p-wave scattering. In order to enhance U, we suggest 
using fermions with large magnetic moments polarized 
in external magnetic field, such as 53 Cr with 6^,3- An- 
other method is to use the p-wave Feshbach resonance to 
enhance U 3> tu. But we do not need very close to the 
resonance, so that U is still smaller than the gap between 
s and p-bands to avoid multi-band effect. 

The p-orbital exchange physics can be conveniently 
represented by using the pseudospin r-vectors defined 

aS Tl = \{plPx ~ p\Py),T 2 = \(p\p y + h.C.),T 3 = 

h{plj)y — h.c), where r 12 describe the preferential oc- 
cupation of orbital orientation, t 3 is the orbital angu- 
lar momentum. Let us first look at the x-bond. Since 
Pz^-orbitals are eigenstates of n with eigenvalues of ±|, 
respectively, this exchange is Ising-like in the absence of 
7r-bonding as H ex (f, r 1 ) = J^Ti(r)ri(r') with J|| = 2fjj/[/. 
Moreover, the Ising quantization axis changes with bond 
orientations. For a bond along a general direction of 




FIG. 1: The r-vector (A and C) and the corresponding orbital 
(B and D) configurations. The p-orbitals in the triangular 
lattices form 45° and 135° angles with the x-axis, and those 
in the Kagome lattices are 15°, 75°, and 135°. 



e v = cosipe x + sm(pe y , we can rotate p^y-orbitals at an 
angle of (p. In the new basis of p' x = cos ipp x + sin tpp y 
and p' y = — sin ipp x ± cos ipp y which are the eigenstates of 
cos 2ipT\ + sin 2(pT2 , the exchange along the bond remains 
Ising-like as 



H ex (r,r + e v ) = J\\[f(f) ■ e 2 J[r(r + eL) ■ e 2( . 



(1) 



The exchange model in the lattice is just a summation of 
Eq. [T]over all the bonds. Although the p-orbital system is 
of pseudospin-1/2, we will take the r-operators as general 
spin-S 1 operators below. 

Orbital ordering appears in all of the square, triangu- 
lar, and Kagome lattices. In the square lattice, Eq. Q] 
reduces to the 2D Ising-model with the staggered order- 
ing. For the triangular and Kagome lattices depicted in 
Fig. [TJ we rotate p-orbitals on each site at 180° around 
the x-axis which transforms the r-vectors as n — > r\ 
and T2,3 — ► — T2 j 3. Correspondingly, the azimuthal an- 
gles (f T of the t- vector and ip p of the p-orbitals satisfy 
tp T = —2<p p (ip p has a periodicity of ir instead of 2ir.). 
Then Eq. [1] along each bond changes to H ex (r, f+ e v ) — 

J\\ [f(f) ■ e v ][f(r + e v ) ■ e v ] for e v = ±e x , ±{\e x ± ^e y ). 
In the triangular lattice, the exchange model can be re- 



organized into H t . 



+J\\ EA T i(r)- S ( S + 1 )} - Thus the classic ground 
state configurations satisfy T^,(r) = on each site and 
[f(f) + r{r + e Vi )\ ■ e lfli =0 on each bond. For two 
neighboring sites i and j, their azimuthal angles ifi and 
tpj of r- vectors should satisfy either ipj = ifi + n or 
ipj = 2ip(iij) — ipj +7r where <p{eij) is the azimuthal bond 
angle. It is straightforward to prove that the only clas- 
sic configurations satisfying this constraint is depicted 
in Fig. U A as the stripe configuration with r-vectors 
aligned along the 90° and 270°-directions up to a 6-fold 
degeneracy associated the lattice rotation group. The 
corresponding orbital configuration is shown in Fig. [T]B. 




FIG. 2: The fully-packed oriented loop configurations in 
which r-vectors lie in directions of ip = ±30°, ±90°, ±150°. 
A) The closest packed loop configuration with all the loops 
in the same chirality. B) The p-orbital configuration for one 
closed loop in A). The azimuthal angles of the p-orbitals are 
45°, 105°, 165°, 225°, 285°, and 345°. 



For the Kagome lattice, the classical ground state con- 
figurations can be obtained by minimizing the exchange 
energy for each triangle. It shows that r-vectors lie along 
the angle bisectors of triangles, which can be consistently 
arranged over the entire lattice as the "Q = 0" state de- 
picted in Fig. [T]C and D. Its ground state is two-fold de- 
generate by reversing the directions of all of the r-vectors. 
Their orbital excitation spectra are gapped in both the 
triangular and Kagome lattices as 1.68JS and 2A5JS 
within a Holstcin-Primakov type orbital wave analysis, 
respectively 

In contrast, the p-orbital exchange model in the hexag- 
onal lattice is markedly different, which exhibits strong 
orbital frustrations. Three unit vectors ei^.3 denoting 



bond orientations are defined as e\ 



62.3 



\e x ± 



n/3; 



Due to the bipartite nature of the honeycomb lat- 



tice, we rotate the p^-orbitals at 180° around the x-axis 
in the A-sublattice and around the in-plane direction of 
e v with cp = 45° in the S-sublattice. This transforma- 
tion changes the r-operators as t\ — > ri,r2.3 — > — ^3 for 
the A-sublattice and t\$ — > —t\$,t<x — » r2 for the B- 
sublattice. The relations between the azimuthal angles 
of the r-vectors and the p-orbitals are ip T = —2tp p for the 
A-sublattice and tp T = it — 2ip p for the £?-sublattice. We 
arrive at 



FeA,i=l,2,3 



3Jn 



5>f(f)- S(S + 1)]. 



(2) 




A similar model is studied for the e 2 g orbitals of the tran- 
sition metal oxides in the 3D cubic lattice 
[2] also has a similar form to the Kitaev model [24 
contrast, the pseudospin r • e; defined here only lies in 
the xy-pl&ne. 

The classic ground states of Eq. [2] require that all the 
r-vectors are in plane and every two r-vectors of a bond 
(ij) have the same projection along the bond direction, 



3 




FIG. 3: The fully-packed unoriented loop configurations in 
which r-vectors lie along the bond directions. A) and C) are 
the r-vector configurations with the closest packed loops and 
the ferromagnetic state, respectively. B) and D) are their 
corresponding p-orbital configurations. 



i.e., the azimuthal angles ipi and <pj of the r-vectors sat- 
isfy <fiU) = ^(0) or <P{j) = fyieij) - <p(i). Clearly the 
ferromagnetic state with arbitrary in-plane polarization 
angle satisfies this constraint. In addition, far more other 
classic ground state configurations can be constructed as 
follows. Let us pick up an arbitrary lattice site i and 
set its r-vector angle ipi — 30°. Then the angle of the 
r-vector on any other site can only take one of the val- 
ues of ±30°, ±90°, and ±150°, thus it is perpendicular 
to one of the three bonds emitted from this site. Let us 
mark the rest two bonds with bold lines, then those bold 
lines form loops with the r-vectors lying tangentially to 
the loops. The ground state configurations are mapped 
into the fully packed non-intersecting loop configurations 
in the honeycomb lattice. These loops are oriented in 
that the chirality of one loop can be changed by flipping 
the directions of all the r-vectors without affecting other 
loops. Fig. [2] A shows one of the closest packed loop 
configurations where each loop goes around the smallest 
plaquette with the same chirality. The corresponding p- 
orbital configuration along the loop is depicted in Fig. 
[2] B. In the ferromagnetic states with polarization an- 
gles of ±30°, ±90° and ±150°, all the loops are infinitely 
long winding around the entire system. Since the al- 
lowed loop configurations are numerous, the system is 
heavily frustrated. It is well-known that this loop repre- 
sentation is equivalent to Baxter's three-coloring model 
[25l dfjl, • If all the r-vectors are constrained to take 
the above six discrete values, these allowed orientations 
just correspond to the six coloring patterns of each site in 
the three-coloring model. Since each loop contains even 
number of bonds, we can assign two colors (e.g. R and 
G) alternatively to bonds along each loop, and the other 



one (e.g. B) to bonds normal to each loop. Each loop 
allows two configurations (e.g. RGRG... and GRGR...) 
representing two opposite chiralities. 

Next we restore the classic picture of the r-vector as a 
U(l) rotor in the xy-plane. Each loop configuration de- 
scribed above has a global U(l) degeneracy associated 
with a suitable arrangement of the clockwise or anti- 
clockwise rotation of the r-vector on each site. For ex- 
ample, for the configuration depicted in Fig. [5] A, this 
degeneracy corresponds to a staggered pattern of clock- 
wise and anti-clockwise rotations on r-vectors in two 
sublattices. For general loop configurations, the rota- 
tion directions of two arbitrary neighboring sites are the 
same or opposite dependent on whether they have the 
same azimuthal angles or not. For each six-site plaquette 
with arbitrary r-vector configurations, we have explicitly 
checked that rotations can be consistently arranged with- 
out violating the ground state energy constraint. Since 
the whole lattice can be decomposed into plaquettes, ro- 
tations can also been consistently arranged in the entire 
system. If we start from one loop configuration and per- 
form a suitable rotation described above at the angles 
of n x 60° [n — 1 ~ 5), we arrive at other five different 
oriented loop configurations. As a result, the classical 
ground state manifold of Eq. [5] is the fully-packed loop 
configurations multiplied by a global U{1) rotation with 
the angle -30° < 9 < 30°. 

If the rotation angle defined above is right at 30° or 
other equivalent angles modular 60°, the r-vector on each 
site is rotated to one of the three bond directions. If we 
mark the other two bonds with bold lines, they also con- 
nect to form loops. For example, after performing the 
rotation of ±90° at A (B) sites for the loop configura- 
tion in Fig. [2] A, we arrive at the configuration in Fig. 
OA. Except a global two- fold degeneracy by flipping the 
directions of all the r-vectors, these loops are not ori- 
ented. The oriented loop configurations with the same 
loop locations but different chirality distributions can be 
rotated into the same unoriented loop configuration. 

So far we have elaborated the large ground state degen- 
eracy at the classic level, which must be lifted by quan- 
tum and thermal fluctuations. For this purpose, we per- 
form a Holstein-Primakov "orbital wave" analysis. We 
consider the two representative ground state configura- 
tions depicted in Fig. [3] A and C, and define that they 
correspond to the 9 = 0° state in their continuous man- 
ifolds with the parameter of the global rotation angle 9. 
We calculate the 1 / S'-correction to the ground state en- 
ergy from the "orbital wave" at arbitrary angles of 9. 
The result is depicted in Fig. 2] and the details will be 
presented elsewhere. The ground state energies of con- 
figurations in both manifolds arrive at the minimum at 
9 = n x 60° (n = ~ 5), i.e., the states represented 
by unoriented loops. Furthermore, the state of Fig. [3] 
A energetically wins over the ferromagnetic state, and 
such a state has an important feature: the appearance 
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FIG. 4: The "orbital wave" correction to the ground state 
energy per site for two manifolds of classic ground states de- 
picted in Fig. [3] A (the solid line) and C (the dashed line). 9 
is the rotation angle. The selected ground states correspond 
to that depicted in Fig. [3] A and its six symmetry related 
counterparts in which 6 = (6 = n x 60°). 

of the zero energy flat band of orbital modes. This can 
be heuristically explained as follows. Let us take an ar- 
bitrary six-site loop in this state. Suppose we perform a 
staggered rotation with a small angle AO only for each 
site along the chosen loop but without disturbing sites in 
any other loop. Only the six bonds connecting the chosen 
loop to outside increase energy. Because the r-vectors in 
these bonds are along the easy axis, according to Eq. [21 
the energy cost vanishes at the quadratic level as 

AE = 6J ll S 2 (A6) 4 . (3) 

Each unoriented loop contributes one zero energy orbital 
(at the quadratic level). The state of Fig. [3] A has the 
maximal number of the zero energy modes. As a result, 
the quantum zero point motion (orbital fluctuation) se- 
lects this state as the true ground state in the large-S" 
limit. We expect that this state not only wins over the 
ferromagnetic state but also the true ground state in the 
large-5 limit, energetically better than any other config- 
uration which always has less number of zero energy or- 
bital modes. Experimentally, this ordering pattern with 
the enlarged unit cell of six sites can be easily detected in 
the time of flight noise correlation spectra. The second- 
order coherence peaks will appear at the reciprocal lattice 
vectors of the corresponding reduced Brillouin zone. 

Next we briefly discuss the effects from the t± term and 
finite temperatures. The t± term generates the orbital 
flipping process as AH(r, r") = J±{ — (r(r) ■ e^.-,)(r(r') ■ 
e^,)+T3(r)r3(r')}, where e~, lies in plane and is perpen- 
dicular to eppi. For the two ground state configurations 
depicted in Fig. [3] A and C, this term favors the ferro- 
magnetic state at the classic level by gaining the energy of 
AE c i — J±S 2 , but pays the cost of the zero point fluctu- 
ation energy around AEfi c = O.OIJ^S as shown in Fig. 
SJ Due to the smallness of the J±/J\\ = t±/tn, AEfi c 
and AE c i are close to each other and lead to rich phase 



competitions. For the realistic system where S = i, 
the plaquette phase in Fig. [3] c is stabilized roughly at 
t± < 0.0Lt|i which can be easily realized in the realistic 
system as calculated in Ref. [17|. On the other hand, 
thermal fluctuations also help to stabilize the plaquette 
state which has the maximal number of zero modes by 
enhancing the entropy contribution. 

In summary, we have presented the general structure of 
the p-orbital exchange physics, which gives rise to many 
different features from the d-orbital solid state systems, 
including the orbital ordering in triangular and Kagome 
lattices and orbital frustration of the 120° orbital model 
in the honeycomb lattice. The six-site plaquette order- 
ing pattern in the honeycomb lattice is found due to the 
"order from disorder" mechanism. Although the above 
analysis was done at the large-5 level, it is well-known 
that quantum fluctuations at 2D usually are not strong 
enough to destroy long range order. It is conceivable that 
the above orbital orderings also extrapolate to the real 
orbital systems at S = 1/2. For example, spin orderings 
in square and triangular lattices of quantum magnets by 
large-S* methods also apply to the spin- 1/2 case. 
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cussions. C. W. is supported by the start up funding at 
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the completion of this manuscript, there appeared an in- 
dependent work by E. Zhao et al. 28] on the orbital 
exchange model but with different conclusions. 
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